Skip to content

docs: add known limitation for non-vision main models via custom API gateways - #2

Open
Ikalus1988 wants to merge 1 commit into
odebo:mainfrom
Ikalus1988:docs/non-vision-model-limitation
Open

Ikalus1988 wants to merge 1 commit into
odebo:mainfrom
Ikalus1988:docs/non-vision-model-limitation

Conversation

@Ikalus1988

Copy link
Copy Markdown

Summary

This PR adds documentation for a known limitation when using CC-Vision with non-multimodal main models accessed through custom API gateways (e.g., corporate internal gateways, model proxies).

Problem

When using CC-Vision with a text-only main model via a custom API gateway, the hook successfully calls the multimodal API and injects text descriptions via additionalContext. However, Claude Code also sends the original raw image to the main model, which may cause errors like "model may not exist or you may not have access to it".

Root Cause

Claude Code's behavior: the hook can add context (additionalContext) but cannot remove the raw image from the API request. This means:

  • With Claude's native models (which support images): works fine
  • With custom text-only models via API gateways: may fail

Changes

  • docs/non-vision-model-limitation.md: Detailed explanation of the issue, root cause, and potential workarounds
  • README.md: Added "Known Limitations" section with link to full documentation

Testing

Tested with Claude Code 2.1.206 on Windows 11 with a custom API gateway (Mify) and text-only main model.

Related

This is a documentation-only change. No code modifications.

…gateways

When using CC-Vision with a non-multimodal main model accessed through
a custom API gateway (e.g., corporate internal gateways), the hook may
fail because Claude Code sends both the text description AND the raw
image to the main model. The hook can add context but cannot remove
the raw image.

This commit adds:
- docs/non-vision-model-limitation.md: detailed explanation of the issue
- README.md: known limitations section with link to full documentation

Tested with Claude Code 2.1.206 on Windows 11 with custom API gateway.
@Ikalus1988

Copy link
Copy Markdown
Author

Dropping in with something useful about the red CI on this PR, since the run is from 2026-08-04 and nobody can read it any more.

The failure is stale, and the content here is fine. I fetched this PR's own commit (514531a, via refs/pull/2/head — the plain SHA is unreachable now, and the branch is deleted) and ran .github/workflows/ci.yml step by step against it:

Syntax check                       ok
--help works                       ok
--test-image w/o API key → exit 2  ok (exit 2)
hook mode w/o new images → {}      ok (prints {})

I ran those on CPython 3.12 and 3.13; the same four steps also pass on main under 3.11. This PR touches README.md and docs/non-vision-model-limitation.md only — no code — so there is nothing in it to fail. The old run's logs have expired and its check-runs are gone (GitHub now reports zero checks for that SHA), which is why it just sits there looking broken.

What I would do: push any commit (an empty one, or a rebase) to trigger a fresh run — that clears the stale red and gives this PR real checks again. If you also want to close a real gap while you are in there: CI never exercises install.sh or the settings.json merge, which are the first two things a user actually runs. I reproduced install.sh in a throwaway $HOME and it works (idempotent merge), so a small CI step asserting that would be cheap insurance, not a bug hunt.

Who I am, and a small offer: I maintain MisakaNet, an open, git-backed library of failure lessons for coding agents (misakanet_search over MCP, no account needed for reads). I only ended up here because I was testing whether that library helps a repo that isn't mine, and this was the most recent failure on the Actions page.

Honestly, for your domain it had nothing to offer: I searched VISION_API_KEY env var not set and hook returns empty JSON silently and got unrelated lessons. The one thing that came out of it is a pattern write-up — a hook whose error handling is deliberately graceful makes "nothing to do" and "I couldn't run" indistinguishable, so the fix is a witness (sidecar log + a stderr line + a CI step that breaks the config on purpose) — filed here: Ikalus1988/MisakaNet#1637

If CI-failure intake is ever useful to you, there are two ways in, both opt-in: call the hosted MCP endpoint from whatever agent works in this repo (https://misakanet.org/mcp, misakanet_search / misakanet_submit_intake), or add one workflow that reuses Ikalus1988/MisakaNet/.github/actions/misaka-intake-bot@main on failed runs and comments the relevant lessons (or files a gap report when there are none). I am happy to send either as a PR — and equally happy to be ignored: no reply needed, and I won't post here again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant